DOCUMENT RESUME \ 


“ED 218 282 _ ™ 820194. 
- AUTHOR Guyette, Susan; Churchman, David 

TITLE Two Models for Ethnographic Evaluation 

PUB DATE . Dec 81 ; 
. NOTE : 18p.; Paper presented at the Annual Meeting of the 


_ American Anthropological Association (Los Angeles, 
CA, December, 1981). 


EDRS PRICE , MFO01/PCO1 Plus Postage. 
DESCRIPTORS | Decision Making; *Educational Anthropology; . 
: : *Ethnography; *Evaluation. Methods; Evaluators; sm 
*Models; *Program Evaluation; Social Science 
Research 
IDENTIFIERS Evaluation Research; *Qualitative Analysis 
‘ ‘ Fe 
ABSTRACT : d 
7 = An ethnographic evaluation adapts data collection and 
analytical methods of the ethnographer to the purpose of 
decision-making. Ethnography and evaluation differ in determination 
of problems addressed’ by the researcher. Ethnographers are interested . 
in slow cultural change, while evaluators. address policy questions : 
requiring immediate answers. The micro-ethnographic evaluation limits / 
the scope of the study to the school, viewed as a subculture. The 
‘resultant data is limited to the perception of a small number of | 
participants. The macfo-ethnographic evaluation considers school | 
interaction. as interrelated with other cultural subsystems. The 
process of enculturation occurs as part of the curriculum. The 
advantages and disadvantages of micro-and macro-ethnographic 
evaluations are outlined. The ethnographic approach to evaluation 
reduces the cultural bias of evaluators, thus making it appropriate 
in explaining programs for minorities. Another advantage of this 
approach is eliminating the necessity’ for a control group. It does 
not require statistical methods, thus making evaluation reports more 
easily understandable. The cultural suitability of the educational | 
program is assessed through’ a macro-ethnographic evaluation. 
Ethnographic evaluations are appropriate when a comprehensive 
understanding of the program and its relationship to society is the 
major factor in judging proyram quality. (DWH) 


. 


RERKRREREREEREEEREEREREEEKERKEEKRRERKEEK EER EEREEEREREKEKEEREEEREREEEEKEEKEK 


* Reproductions supplied by EDRS are the best that can be made ' * 
* 


from the original document. = 
eet iT eri titi titi litt rititiiettititisitiiettitsti iti titi iret ere er 


. 


w 


£p218282 


i 


* 
aa ae aS 
PERMISSION TO REPROBUCE THIS 52% Be 8 

| : MATERIAL HAS BEEN GRANTED BY 8 z 3 3 2 

F \ | : & g 3 3 H 

Mb ‘ wad 

| seeeli 
ee , : ies Egil, fe 
TWO MODELS FOR’ ETHNOGRAPHIC EVALUATION poe. '. 23 

TO THE EDUCATIONAL RESOURCES es°RE.'s 
INFORMATION CENTER (ERIC) ; 4 $ g ] g 7 gs : 
| nf os 425 rie?) 
Susam Guyette (University of California, Los Angeles) 23 % ae 

: David Churchman (California State University, Dominguez Hills) 
: ‘I. INTRODUCTION ; 
° vt = , ° \ 


Tw £r0 s3Y 


Se 


. 


In recent years, educational anthropologists have turned 
attention toward the techhiques of program_evaluation. The 


“ 


purpose of evaluation.,is to*measure ae toward the 


“accompl ishment of goals. - AS an applied discipline, evaluation 


ultimately assists educators and administrators to make 
decisions regarding program improvements or to become aware of. 
program results, thus enabl ing them to make informed judgements 


about program results. ‘During the past decade dbesappointnent 
ar) 


with evaluations often focused upon the tendency toward cultural 


bias exhibited by the mire frequently dom itfant-cul ture 
evaluators (Sjoberg 1975: 43). That isthe formulation of 
evaluation questions, the interpretation of goals, and methods 
of measuring, progress: vare particularly susceptible to cultural 
skew. With this: ‘pat éfeul ar prebi em in oe educators and 

Kin iblaknators Hews, with increasing frequency, called on 


SOShreRe OD see to participate in this form of: contract Pi 


research. ves: 4 f 


The majority of.-ethnographic: evaluations. conducted, thus far by, ~ 
‘dueationsl anthropol ogists are classroom ethnographies, which f 
focus on the process of classsroom interaction. Al though this! 
emphasis answers" ‘important questions concerning what actual l ys 
happens in the classroom setting, it is limited for decision 
making applications Use of the wide range of evaluative 4 


techniques can’broaden the kind of results suitable for. f eo 


- ‘tion, Los AAgeles, CA, December 1981 
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applied seitings. Then, two models 


ethnographic evaluation. As an inffroduction, we will discuss a 
If 


brief history of egenetior: outlining the successes 


and ares of past eshinigineyy It is from this point that new 
iboed for the culturally sensitive 


“pvetuation.* > ae a 


- 


A ¥ . 
Program evdluation bec important, jin terms of resources 


utilized, responsibil itie#//undertaken and results expected, as a 
consequence of the socia {program experiments initiated during 


the "Great Society” era the 60's. Essentially, program 

i : cs 

evaluation was the at by which the Congress expected to 
‘ 


maintain aceon iiny | pexens results and identify successful 


Se 


‘The initial sttenot om carry out hese responsibil,ities 
relied on traditionat esearch _desyaen descended from the methods 
qriginated by Ronald | sher in the late nineteenth and early 
twentieth centures. ese methods relied on randomization, 

E éontrol groups and st istical analysis af data for the purpose 
of inferring causal relations between independent and’ dependent 


vartables. They had ‘Been progressively refined through 


“subsequent “experimental work, primarily in university ee es 
laboratories, and had recently become capable of handling large 

- amo ie of data with) he advent of the computer. , 

The educational pr grams initiated during the Great Society, 


more than most of the other experiments of the period, had the 


additional advantage o f an, established sraditjon for measuring 


effects. This tradition was based on EN work of Edward 


, Thorndike, Lewis Terman, «Janes Cattell wad others. The emphasis 


was on the use of paper and pencil tests administered; to ° 


indivfdual studénts and summarized for individuals, classes and 


higher groups through descriptive statistics. 


Evaluation, based on these traditions required random 
assignment of students to an experimental pregean or usually to 
an existing educational arBiean hs a control; testing ait 
students to generate data at predetermined fF points in time; and 
statistic \ tests to determine ly eka success of the two 
programs. Viromcry atl the evaluations done fon the following 
decade ene variations on this basic’ scheme, 

Results were disappointing. Random assignment frequently 
proved impossible in ee Haat éateings; and in almost all of the 
social programs of the period. Tests used to collect data were 
predisposed to favor either the experimental or the control ‘ 
group, making interpretation of results unexpectedly difficult. 
Insufficient attention wa iven to verifying the extent to 
which ‘the experimental (a was carried Gut, leaving 
amb iguous\exactly what had to be done to achieve the same 


1 and 


results again. Halo effects, Hawthorne effects 
contamination of the experimental program by the control and 
vice-versa further complicated interpretation of results... 

Knowl edge that an experimental program was more successful than 
the existing program at some sertioular school was inadequate in 
making a decision about the ‘exper imental program unless the 


regular program at that school was identical to the control 


“group program; even if the experiment went well, the evaluation 


did not provide very useful information. But, it did overwhelm 


ar ® 
people untrained-in the traditions of Fisher and Thorndike with 


- 


the appearance of aeeipuey and objectivity, a consequence of the 


tables and numbers to several dec inal points, which dominate the 
reports, Tests also often were found to be culturally biased 
when used to measure progress of minority students who were the 
major recipients. of Great Society assistance. 


More important than these technical difficulties, the 


administrators for whom these evaluations were done often were 
dissatisfied with the results. ‘The txotcal evaluation described 
above, eee out at great expense, usually *may be ae 
to the single conclusion; that there is, or jis not, a 
statistically significant difference between two educational 
programs. Typically, the educational significance of this 

statistical ST ATEP ENS TS not eae comparable 

"obfective” standards do not exist and the conclusion would have 

to be reached “subjectively", anathema to the tradition; yet 

this is the most important question for the administrator. The 

approach denen tind shave leaves totally unaddresssed other # 
equally vital questions, such as the costs of the experimental 


program, or .the way different groups in the community would — i 


react to its adoption, or the relationship of the experimental 


~ 


program to’ the school curricul upmas a whole. 

Difficulties, such as these with traditional methods for 
conducting evaluations -are among the reasons many have ‘argued 
that broader, richer methods must be substituted. Among the 
many that have been suggested is ethnography, which uses a wide 
range of both quantitative and qualitative methods. 
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III. DIFFERENCES BETWEEN ETHNOGRAPHY AND EVALUATION | 


There are some important differences between ethnography and 


evaluation that should be recognized before the two disciplines 


« 


can be integrated, The most important of these differences is 

in “determining the “problem that is addressed by the researcher. 
e Ld ‘ 

pave Aes are interested in culture, whi¢h 


changes slowly; evaluators address policy questions that 


S| 
. 


typically require answers without delay. Ethnographers, under 


little or no time pres ure, typically allow the issues to’ ble 


studied to emerge over ketatively long, periods of time. 
Evaluators not only work to specific short-term deadlines, but 
must address questions ra\ised by someone. eise. Thus, 
‘ethnographers expect to learn the nuances and meanings of a 
cultural evuteA after yearns spent gaining rapport with people in 
the culture being studied. Evaluators get in, get the best 


information they can and get out quickly. Ethnographers are 


-process-oriented, while evaluators concentrate on outcomes. 

One consequence of these differences is that ethnographers 
often feel that their science is being exploited by evalUators 
who are using their techniques «in new ways for new purposes. 
But, man has a long history of modifying existing tools to 
achieve new purposes. “Adapting ethnographic methods to 
evaluation studies requires selection of the retevant elements 
of traditional ethnographic methods and ignoring others not , 
relevant to evaluation tasks. Jhe plethora of names such as 
"survey ethnography" and "contract ethnography". may represent 


recognition that adaptation rather than adoption is taking 


place. ? 
In this view, the. fundamental question is whether the / 
traditional methods of ethnographic data collection and analysis 


can be adapted to conducting an evaluation that’ better f 
meets the needs of a.program administrator than does the more 


common experimental method. Traditionally, ethnographers make 


inferences from what people say, what people do and what people i 
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use, - They commonly rely on basic tlbols.such as the interview, 
the questionnaire and observation, | Evaluators employ these 
basic tools in addition to ratings or rankings (by staff, 


participants, or managers), journals kept by staff, case 


histories, organizational record. keeping, analysis of documents, 
clinical examinations, information Leet: eetadardised attitude 
inventories, and ability tests. - Although the etlinographic 
evaluation. efforts of the past have utilized primarily the © 


former, and the woe inen val deBign draws primarily from the 


latter ewohadauee, combinations of sat Feats and quantitative 


techniques may provide'the basis for future ethnographic 


evaluations which both meet decision-making needs and are. 
culturally sensitive... _ 8 
. ; . 2 : 
IV. ISSUES IN EVALUATION. DESIGN! Pa 
Design of an evaluation involves a large number ‘of often 


-contradic.tory considerations which interact with one another. 


The inevitable consequence is phat tradectis must be made among 


, design factors,and compromise's results in“less than ‘per fect 


studies. But, it is vital that the evaluator understand what 
these tradeoffs and compromises, are so that. the best possible 
evaluation is provided ‘in’a specific tinue: This section 
of the paper discusses six such design issues. 

A first design decision concerns. the audience for whom the 
evaluation is intended. ‘The aud tance for an evaluation consists 
of decisonmakers who face a specific deadline. These 
decisonmakers may be divided into those "internal "to the 
organization running. the program being evaluated (e.g. program 
staff or administrators), andi those "external" to the 


organization (for exampte, a funding agency or legislative 


e 


body). The former usually want help in improving\ their program 


and some evidence that they are doing a good job.  (Jhe*latter 
usually want to know such things as whether the program should : 
continue to receive funds and if so at what level. The term 

"formative. evatuation™ often is used to distinguish the type of 

evaluation done for staff and administrators, while-the term 4 
"summative evaluation” is used for that aimed at funding 
agencies or legislatars. We will discuss these terms below more 
*ubhy, Internal- evaluation has the advantage of a much greater 
iauereearedner-ot organizational dynamics, history, : 


’ 


personalities, capabilities and the like, and therefore often 


results in a more sensitive evaluation. On the other side of 
the coin, unless the organization has evaluation specialists a 
available, the evaluation may not be completed properly or on. 
time and the results may be less credible to sevanwal “alidventeas 
External -evaluation has alweet precisely the opposite advantages 
and disadvantages. AS a general rule, formative evaluations can 
be conducted iqternally, but summative evaluations should be 
conducted by an outsider. 

A second issue concerns the application of the evaluation, 
for purposes of decision or judgement. Et hnographers. are 
expected to avoid making a judgement about tke society studied; 
even less acceptable might be the idea of otfering suggestions 
as to how to make improvements. Hav ing accepted this 
philosophy, they may have some difficulty accepting the applied 
character of evaluation research, which is aimed, ultimately, ut 
judgement and decisionmaking. Within evaluation circles, the 
debate’ is whether the @vafuator or the employer should draw 
conclusions, and whether these conclusions should be |imited to 
practical decisions or to more comprehensive judgements. It is 


a perfectly acceptable position to insist that you, as an 


. 


ea oe 


evaluator, will not include recommendations or judgements in 
‘your evaluation, but it would be ‘inappropriate to accept a 
contract to provide an employer with information needed to make 
judgements or decisions and not to provide that information. 
Further, it is important that you understand the purposes for 
which the évaluation“are to be undertaken before, acceptina the 
contract or designing the evaluation. As suggested, these fall 
into the general categories of (1) prévision of information 


required to make specific decisions and (2) provision of 
information required to make jddaeaanes about the program. 

A third choice facing the evaluator during the formulation of 
.the evaluation design concerns the evaluation question. This 
step in problem definition is perhaps one most likely to ge 
influenced by cultural perception. Ay evaluation that 
will meet the jiterwarion needs of an educational administrator 
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. ‘ . 4 . 
may provide answers to evaluation questions®™ such as : 


* What work is done in the program, being evaluated--that is, how 


do students and teachers actually spend their time? 


“* How is "individualized’ instruction" experienced by students? 


(versus how well does individualized instruction work?) 


* How do the various types of work don® in the program relate to 
~ N. e . 


one another? 


* What are the major stages through which the program moves, and 


what work is done within each stage? 


* How 1S achievement measured or identified and what status or 


rewards does it provide? 


.* How is the program administered or controlled? 


fra. 
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* How does the program respond to the cultural differences or, 


similarities of the culture-at-large? 


* How do the activities in thg program relate to the cultural 


activities in the community? 


As the questions asked during any evaluation will depend on 
the decisions that have to be made, this list is illustrative, 
suggestive and incomplete. The task of the see cater is fo help 
define the necessary questions, then to coljJect the necessary 
-information and to analyze it. The actual decisieun are . 
necessarily the task of the administrator. This is consistent 


with definitions that view the evaluator as a collector, 


--% 


analyzer and interpreter of in-formation, and recognize that the 
authority for-decisions must remain with the administrator who : 
is responsible for the program (e.g. Alkin, 1969). 

‘A fourth design issue concerns type of evaluation appropriate 
to the eval\uation question. One of the most fundamental’ 
theoretical distine tiens wade by evafuators is that between 
formative and summative evaluation (Scriven, 1967). The purpose 
of the former is improvement; chat o f the latter is 
certification or val idation, Formative evaluation implies 
identification o f elements of an gnaeine program that are not , 
working as well as they etahts so that improvements can be made. 
Formative evaluation is an ongoing process, throughout t'he ‘ 
development of a project. Data are collected while the program 
is in process, to determine the continual effectiveness of the 
program and make changes in the program activities if necessary. 
Using this approach can Baye resources and improve results, by ’ 
"forming™ the program as effectiveness is determined. Formative 


evaluation shquid, logically, be carried on until the cost of 


conducting the evaluation exceeds the value of improvements that 


cam be made in the program. Summative evaluation occurs after 


the program is final or completed, although pretesting of the 
data collection instruments and the data sok ieation usually 
occurs while the program is in progress. The program then 
should be carried out in a homeostatic or unchanging state. The 
results of a summative evaluation are often used in making 
decisions as to keeping a particular program structure for * 
future use. ‘The summative etalietien f¢ particularly useful in 
comparing different types of oreatans and deciding which type is 
the. most effective. A ounwdcive evaluation often enables 


program administratérs and funders to decide whether the program 
: heuld be adepted permanently or in some wider context. 

The theoretical distinction is’ fairly clear, but in practice 
many evaluations claim to be one (for example, summative) but 
include information Hel Svant tothe other (for .example, 
recommendations Un how to improve a aReg end « One reason for 
this state of attained js the extent to which evaluators eee aA 
traditional research design. Logically, different questions 
require different wathods, but this has been rare in practice. 
The diet iperion between formative and summative evaluation 
should be kept in mind in selecting the methodology for a 
particular eval fation problem. 

A fifth issue addresses the methods -of goal-based versus 
goal-free evaluations. The overwhelming majority ‘of evaluations 
are conducted by measuring the extent to which the intended 
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objectives of the project are being achieved, but some writers 
baee acquea ehae the goal-based method gives the evaluator’ 
"tunnel vision" that leads to overlooking important side 
effects. These effects could be desirable or not.’ For example, 
a project to teach children to read may awaken their interests 


in whatever the subject matter of re reading material is --- 


ci a? 


a 


. 


endangered species or computers or art history or whatever. 


Usually this would. be considered a desirable unintended effect. 


If it. were’ missed by an evaluator who only measured reading 


skills, then the value of the program woutd be underestimated. 


The ieee a: method differs from the goal-based in that the 


_ objectives are: not ‘known, by the evaluator at the start of the 


evaluation. They. ‘are discovered or sabes participants 


dur ing’ the evaluative process. Goal-free ayel uation hae 


“potential for use with ethnographic methods such as-participant- 


observation and demain analysis. Also, the goal-free approach 
May be more appropriate in minority communities where the 
implemented goals may differ from those defined by dominant 
‘culture odiearares, Whenever. unintended effects might 
significantly influente the decisions to be made about the 
project being evaluated and there is reason to suspect that the 
effects cannot be predicted, goal-free evaluation should be 
considered, either as an alternative or as a ‘supplement to the — 
more’ traditional goal-based evaluation. ‘However, the choice: 
should not be made Nightly.” By their very eature unintended ; 
effects are difficult to detect, “particularly when it is . 
realized that they may ‘take place beyond the confines of the- 
project. “Further, there are ‘no. ERE accepted methods for 
conducting. “goal-free’ ‘eval darted, land. participants may perceive 
the process as very threatening because they have no idea of 
what the evaluator is looking TOP < 

| A sixth issue’ involves decisions regarding data. Throughout 
the heyday of evaluations conducted of Great Society programs, 
there was a distinct preference for quantitative data. It was 
seen as more reliable and thus more valid. ! It was more 


“objective”. More sophisticated and wensitive statistical 


“techniques were available for analysis of quantitative data. 
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«But, results did not live ig to expectations, Gradually, . ya 
criticism grew, Rel iakility and validity were gatnee ay oy ; “wa 
’ measuring variables that were easy rather than important. to 
measure. “objectivity was compromised, due to factors such. as. 
cultural bias. The inability of refined techuiausa to find - 


ms 


differences, between. radically different programs suggested that 


aa 


~ Statistical and educational significance were quite different. get ae 
from one another. Led by Robert Stake, the statistical \ 
evaluators began to. question the appropriateness of their ‘ 
- methodol ogical, assumptions, and then to advocate» ‘so-called . 
“5 ' qualitative methods. Advantages of the ‘qualitative approach are 
that +t. can be discovery-orjented, exploratory, descriptive, 
inductive, process-oriented, and’ holistic. More recently, 
pn ey “evaluators and: ethrographers have adapted qualitative techniques 
for ethnographic’ eval uations, The consequence has been £o- ace” 
: ‘vastly extend methods eonsidkrad acceptable- fer conducting 
| - | evaluations. Now, ‘instead ‘of blindly choosing ‘quantitative data 
nate gained through a traditional research design, phe: evaluator must 


. awkes a ational choice. fron among many. seesibilities: ‘ind de fend 


’ e 


.the choice’ against ‘all others as -the most appropriate for. a 


apticular circumstance. 
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.; Vs. TWO; MODELS FOR ETHNOGRAPHIC EVALUATION. 


sie | 
; To the present, educational anthropologists in,applied © 


ws 


settings usually adapt evaluation techniques while conducting . 4 


.classroom ethnography. The potential use of evaluative 


. , techniques extends. beyond this scope; and. ‘to- illustrate an 


alternative model, for ethnographic eval uation the distinetion. 


f , 
; a between Breracane macro- ethnographic ‘eval uation. is. developed. A “Ss 
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ial The micro= ethnographic. .eval uation limits the scope of the 
; study ta the school . Supporters of this approach (Wilson 1977) 


view the. school asa subculture, utilizing Senn earaei hs methods 


er. to analyze and interpret activities according to the i 
i participant* $, reality. The resulting eroduet of this evaluation 
is generally an. ethnography describing the program activities, e 
who interacted with whom, and the participants’ interpretations. 
ae The product may address the question, "Did the program 
accomplish its goals?" by discovering the goals within the 
school considered as a closed system. One common accomplishment 
of the school ethnography is the possible discovery of Ee 
. ‘discrepancies between the ideal or stated behavior and the real, 
“or actual behavior as reflected in activities. Through 
, ethnographic interview within the school, it is possible to ask 
| participants if activitiés in the school are cohnected to : 
7 » elements of the cul ture as a whole. This data is limited, 
| however, to the perception of a small number of participants. 


bs . In contrast, One macro-approach to ethnographic evaluation 


considers school “interaction as interrelated with other cultural 
subsystems (Wolcott 1975; Everhart 1975). In other words, the 
school is not viewed as a closed system, but rather as a part of 
' "the culture as a whole. This approach takes into account the 
. process a enculturation, whereby the transmission of that . . 
cultural knowl edge shared by the community occurs as part of the 
curriculum- “There are three major steps to condustine the macro 
: lgeunal yates First, a study is conducted to determine how each of 
. the cultural subsystems work in the community. This involves 
observing, participating, and interviewing to identify 
activities. The cultural meaning at aceiviete is identified 
j : , and domains of 


e . or] 
e _ finteraction are identified for the culture. Second, a domain 
an 
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analysis is conducted with data obtained from activities within 
the school. And thirdly, the final analysis compares the 
domains and world view of the culture-at-large with the 
transmission of knowledge in the school. Similarities and 
differences, are then identified through this comparison. When 
siatunet iene exist, the more common consequence is community 
opinion that the school does’ not reflect the social. proces'ses of 
the culture. | ‘ , 
Advantages of the mi¢ro-approach over the macro-approach 
include lowered costs, reduced time, and narrowly defined 
pouKdGriees. Disadvantages of the micro-approach are that the 
study is more likely to be biastd by the goals set by the 
educators and the scope of the study is limited. The major, and 
evitranely important, advantage of the wieeouaparoach is the 
discovery of discrepancies between the educational goals of the 
community and ‘the actual activities within the school. For this 
reason, the macro-approach is more suitable to non-Western 
cultures «in parnjcul ar. Where time and funds may pose 
constraints upon the macro~approach, existing ethnographies can 


supplement ethnographic fieldwork in the culture-at-large. 
‘ Th@re is an important advantage ofthe aacre-aparoach over 
the i: Cees that should be considered. With the micro- 
approach, if the school is studied as a subculture, then 
connections to the culture-at-large are minimal. In the macro- 


é 


approach, when URNS eee first, followed by an 

_ analysis of the activities inthe school, two types of. . 
information arte discovered. It is possible to see if there are 

No connections between activities in the culture and the — 


activities in the schools, as well as if the activities in-the 
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schools’ do “not connect to the: culture. This information ae e ee eon 4 
be ° . . 
eAsiart “for determining if) items need to.be ddded to tte ee | 


' it _ = 
curr+cul um to reflect elude ‘cultural. fearninds or if there are; es 


™~\ 


‘items in the curricul um that need to be deleted. The oses co 
t 

of fhe syn uation, advantages, and disadvantages of each : 

approach should be weighed in dectding upon the more appropriate 


ethnographic evaluation. \ . , . 


‘ 
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[VEL SUMMARY. * | : 7 


An’ ethnographic: evaluation adapts the data collection and 


analytical methods ‘of the ethnographer to the purpose of 
decision-making. The two. most important of these adaptations 
are advance specification of the questions to be addressed and 
specific, usually,eminent, deadlines that preciude the 
development of rapport vouelt y desired by ethnographers. Such a 
summative evaluation remains faithful to the traditions of 
ethnography in attempting to understand educational - programs 
from the viewpoint of the, participants. Interpretations probably .e 
will emphasize description of program progressgand structure. 

rather than how much course content has been absorbed by . 


: students or what their attitudes ate toward the subject. One 


‘advantage of taking the eehfegiraphys approach is in reducing the 4 
cultural baggage, or the dah ube that the evaluator carries with 
hia into the cultural setting. these val iied sek uae bias 
into the formulation of the evaluation questions or into the 
final recommendations. The approach presented is therefore 
particularly. appropriate. in explaining programs for minorities F 
within any culture. 

The ethnographic approach has the second advantage of not J ’ 


requiring a control .group. Judgements can be reached without * 


ho 
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having to set up or ex tain to parents why children are 


receiving different programs, or in situations where the total 


“number of students does not permit such division. If the purely 


qualitative methods we have suggested are the only ones used, 


et 


the judgement must’ be A eras achievement data 


characteristic of most evaluations. This may not-be the 


oa 


¥ “\; 
weakness it SADE ES oe previous are suggests that one can 


estimate progran effects ust as well thon time on task as from 


_tests. Second, testing poses special problems of cultural bias 


particularly ner minority students. Third, the statistically 


- 


signifi¢ant r dsultas reported from control-group evaluations 
often seem toc seat] to be educationally -significant, so have, P 
little impact on the overal | judgement made anaue a program, 

A third “sivantage of: the - qualitative &thnographic Ae we 
have waa is that it doesnot requirg the often esoteric 


statistical methods. The result is an evaluation report that 


_the administrator who seldom understands these methods ‘does not 


have to take lon faith. Thus, the reports themselves are more 
useful to the people for whom they‘are intended. 


A fourth advantage of the ethnographic approach is that the 


activities and views of the culture-at-large are | inked to the 


activities within the school. This enables the discoveries of 
linkages and disjunctions in the educational process, providing 
direction for revision of curriculum content. The cultural 
appropriateness, of the educational program is assessed through 
eat the macro-ethnographic evaluation. 

Tt appears then that ethnographic methods may be useful for 
the task of conducting evaluations. They are particularly 
suitable when a comprehensive understanding of the program and 
its retationship to society is the most important factor in 
judging program quality, as opposed to such other bases as cost 
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or cognitive achievement. But, the decision-making purpose of 
evaluative cesearch requires adaptation rather than adgggion of 


traditional ethnographic methods. ~ - 
" ; | . Ae 
‘NOTES: 2 #-9 a 


Pr . No 


sae Hawthorne effects concern bias introduced by the subject 
through knowledge of inclusion tn‘the experiment; whereas, Halo 
effects are introduced when;the researcher or teacher is biased 
due to prior impressions of the student. 


Rs "Evaluation questions" defjne the purpose of the study; 
; "interview questions" are asked of participants selected from 
' the program being evaluated. 
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